Image sensing device for correcting image using block areas and method of operating the same

ABSTRACT

Provided herein may be an image sensing device and a method of operating the same. The image sensing device may include an image sensor configured to acquire an image including a plurality of pixel values, a memory configured to store reference gain values of each of a plurality of block areas included in the image, and an image processor configured to calculate gain values included in each of the plurality of block areas using the reference gain values and to output a correction image in which the reference gain values and the gain values are applied to the plurality of pixel values, wherein the block areas include a first block area and a second block area having a shorter distance from a center of the image than the first block area and having a size greater than that of the first block area.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority under 35 U.S.C. § 119(a) toKorean patent application number 10-2021-0059457 filed on May 7, 2021,the entire disclosure of which is incorporated by reference herein.

BACKGROUND Field of Invention

Various embodiments of the present disclosure generally relate to anelectronic device, and more particularly to an image sensing device anda method of operating the image sensing device.

Description of Related Art

An image-sensing (image sensing) device is a device which acquires animage. Recently, with the development of the computer industry and thecommunication industry, demand has increased for image sensing devicesin various electronic devices such as a smartphone, a digital camera, agaming console, an Internet of Things (IoT) device, a robot, a securitycamera, a medical camera, and an autonomous vehicle.

The image sensing device may correct an image to improve image quality.The reason for this is that the quality of an image may be degraded dueto the pixel structure of the image sensing device, wavelengthcharacteristics of light, fabrication error in a lens, alignment errorbetween the lens and an image sensor, or the like. In particular, whenan imbalance between green channels (e.g., a channel Gb and a channelGr), which correspond to one of the colors most sensitive to humanvision is increased, a factor that degrades image quality, such as gridnoise, may occur in an image.

The image sensing device may allow data that is to be used forcorrection to be previously stored in a memory. However, an optimizationtask is required for minimizing the resources required for dataprocessing and storage while maintaining performance, such asimprovement of image quality, in that resources such as hardwareresources are limited.

SUMMARY

Various embodiments of the present disclosure are directed to an imagesensing device and a method of operating the image sensing device, whichmay reduce the computational load and the amount of data to be used forimage correction.

An embodiment of the present disclosure may provide for an image sensingdevice. The image sensing device may include an image sensor configuredto acquire an image including a plurality of pixel values, a memoryconfigured to store reference gain values of each of a plurality ofblock areas included in the image, and an image processor configured tocalculate gain values included in each of the plurality of block areasusing the reference gain values and to output a correction image inwhich the reference gain values and the gain values are applied to theplurality of pixel values, wherein the plurality of block areas includea first block area and a second block area having a shorter distancefrom a center of the image than the first block area and having a sizegreater than that of the first block area.

An embodiment of the present disclosure may provide for a method ofoperating an image sensing device. The method may include, calculatingreference gain values of each of a plurality of block areas included ina calibration image acquired through the image sensing device,calculating gain values included in each of the plurality of block areasusing the reference gain values, storing the reference gain values andthe gain values in a memory, and when an image is acquired through theimage sensing device, outputting a correction image in which thereference gain values and the gain values that are stored in the memoryare applied to the plurality of pixel values, wherein the plurality ofblock areas include a first block area and a second block area having ashorter distance from a center of the calibration image than the firstblock area and having a size greater than that of the first block area.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an image sensing device according to anembodiment of the present disclosure.

FIG. 2 is a diagram illustrating an image sensor according to anembodiment of the present disclosure.

FIG. 3A is a diagram illustrating a pixel array according to anembodiment of the present disclosure.

FIG. 3B is a diagram illustrating an image acquired through an imagesensor according to an embodiment of the present disclosure.

FIG. 4 is a diagram illustrating an image processor according to anembodiment of the present disclosure.

FIG. 5 is a diagram illustrating a calibration image according to anembodiment of the present disclosure.

FIG. 6 is a diagram illustrating a gain map according to an embodimentof the present disclosure.

FIG. 7 is a diagram illustrating any one block area according to anembodiment of the present disclosure.

FIGS. 8A to 8D are diagrams illustrating a method of calculatingreference gain values according to an embodiment of the presentdisclosure.

FIGS. 9A and 9B are diagrams illustrating a method of calculating gainvalues according to an embodiment of the present disclosure.

FIG. 10 is a diagram illustrating a correction image according to anembodiment of the present disclosure.

FIG. 11 is a flowchart illustrating a method of operating an imagesensing device according to an embodiment of the present disclosure.

FIG. 12 is a flowchart illustrating a method of operating an imagesensing device according to an embodiment of the present disclosure.

FIG. 13 is a diagram illustrating a computing system including an imagesensing device according to the embodiment of the present disclosure.

DETAILED DESCRIPTION

Specific structural or functional descriptions in the embodiments of thepresent disclosure introduced in this specification describe embodimentsaccording to the concept of the present disclosure. The embodimentsaccording to the concept of the present disclosure may be practiced invarious forms, and should not be construed as being limited to theembodiments described in the specification.

FIG. 1 is a diagram illustrating an image sensing device 1000 accordingto an embodiment of the present disclosure.

Referring to FIG. 1 , the image sensing device 1000 may be operatedunder the control of a host 3000.

The image sensing device 1000 may acquire an image in response to arequest from the host 3000. Further, the image sensing device 1000 mayoutput the image to the host 3000 or a device indicated by the host 3000in response to the request from the host 3000. Here, the deviceindicated by the host 3000 may be a memory device which stores data or adisplay device which outputs data using a visual method.

The image sensing device 1000 may be implemented in the form of apackaged module, a part or the like. In this case, the image sensingdevice 1000 may be installed in the host 3000. Alternatively, the imagesensing device 1000 may be implemented as an electronic device separatefrom the host 3000.

The host 3000 may be implemented using any of various electronicdevices. For example, the host 3000 may be implemented as a digitalcamera, a mobile device, a smartphone, a personal computer (PC), atablet PC, a notebook computer, a personal digital assistant (PDA), anenterprise digital assistant (EDA), a portable multimedia player (PMP),a wearable device, a black box, a robot, an autonomous vehicle, or thelike.

The image sensing device 1000 may include an image sensor 100, an imageprocessor 200, and a memory 300.

The image sensor 100 may acquire an image Img. In detail, the imagesensor 100 may acquire an image when a command for controllingacquisition of an image is received from the host 3000. The image mayinclude a plurality of pixel data pieces (or a plurality of pixel data).Independent pieces of pixel data may be mapped to respective pixels.That is, the image sensor 100 may acquire an image including a pluralityof pixel data pieces by acquiring the pieces of pixel data correspondingto respective pixels. The pixel data (or the pixel data piece) mayinclude information indicating at least one of a location, a colorchannel, a pixel value, and an exposure value.

The location of a pixel may indicate the location at which thecorresponding pixel, among the plurality of pixels, is arranged.

The color channel of a pixel may indicate the color of light for thecorresponding pixel. For example, the color channel may include one of ared channel, a green channel, and a blue channel. The green channel mayinclude a first green channel and a second green channel, which areclassified depending on the location of the pixel. For example, a pixelwith the first green channel may be a pixel in the same row as the pixelwith the blue channel and the pixel with the second green channel may bea pixel in the same row as the pixel with the red channel. In otherexamples, the pixel with the first green channel may be a pixel in thesame column as the pixel with the blue channel and the pixel with thesecond green channel may be a pixel in the same column as the pixel withthe red channel.

The pixel value may indicate the brightness of light for thecorresponding pixel. In an example, it can be seen that, as the pixelvalue is greater, the light is brighter. The exposure value of a pixelmay indicate the time during which light is sensed for the correspondingpixel.

For this, the image sensor 100 may be implemented as a charge-coupleddevice (CCD) sensor or a complementary metal oxide semiconductor (CMOS)sensor.

The image processor 200 may correct the image. In detail, when an imageis acquired through the image sensor 100, the image processor 200 maygenerate a correction image Cor_Img in which a gain map is applied tothe image. For example, the image processor 200 may generate acorrection image in which gain values included in the gain map areapplied to pixels. Also, the image processor 200 may output thecorrection image. For example, the image processor 200 may output thecorrection image to the host 3000 or the device indicated by the host3000.

For this operation, the image sensor 100 may acquire a calibration imageCal_Img. The calibration image is an image for extracting the gain mapthat is used to correct the image acquired from the image sensor 100.For example, the calibration image may be an image acquired by the imagesensor 100 to capture a white background in an environment havinguniform illuminance. Due to the structure of the image sensor 100 or thewavelength characteristics of light, distortion, such as non-uniformityin a pixel value for each pixel location or each color channel, mayoccur in the image acquired through the image sensor 100. Becausedifferent types of distortion may occur in respective image sensors 100,separate gain maps may be required for respective image sensors 100. Thecalibration image may include a plurality of pixel data pieces. Thecalibration image may have the same pixel data array as the imageacquired from the corresponding image sensor 100.

Each gain map may include a plurality of gain values. A gain value maybe a parameter for correcting the pixel value of the corresponding pixeldata piece and may indicate a specific numeric value. The gain map maycorrespond to the size of the image acquired from the image sensor 100or the calibration image. The size of the image or the calibration imagemay be represented by the number of pixel data pieces arranged in acolumn direction and the number of pixel data pieces arranged in a rowdirection.

For example, when the number of pixel data pieces included in the imageor the calibration image is M×N, the number of the plurality of gainvalues included in the gain map may be M×N. Here, M and N are naturalnumbers. That is, the number of gain values may be the same as thenumber of pixels. Here, the plurality of gain values included in thegain map may correspond one-to-one to the plurality of pixel data piecesincluded in the image. Respective gain values may correspond to thelocations of respective pixels.

The number of data bits to be assigned to each gain value may changewith the location of the corresponding pixel. The larger a number ofbits that are assigned to data, the wider a value range of the data is.For example, when calculating a gain value to which a small number ofbits are assigned, the calculated gain value may be expressed within asmall value range allowed by the small number of bits.

The image processor 200 according to an embodiment of the presentdisclosure may calculate reference gain values included in the gain mapusing the calibration image. In detail, the image processor 200 maycalculate reference gain values corresponding to vertices of each of aplurality of block areas using the plurality of pixel data piecesincluded in the calibration image. The calibration image may include theplurality of block areas. The block area may be defined by a pluralityof pixel locations. Accordingly, some gain values included in the gainmap may be calculated. The image processor 200 may store the referencegain values in the memory 300.

The gain map may include a plurality of block areas. The gain map mayinclude a plurality of gain values. The plurality of gain values mayinclude the reference gain values and normal gain values. Each of theplurality of block areas may be a rectangular or square area that isdefined by plural pixel locations. However, this is only an embodiment,and the plurality of block areas may be modified into respective areashaving various shapes.

Each block area may include gain values corresponding to the locationsof pixels. Each block area may include reference gain values. Thereference gain values are gain values corresponding to some of theplurality of gain values included in the gain map. Respective referencegain values may correspond to the pixels at preset locations. The presetlocations may be locations corresponding to the vertices of each blockarea. For example, the block area may be defined by 4×4 pixel locations(or pixel values) ranging from (1, 1) to (4, 4). The block area mayinclude gain values corresponding to respective pixel locations from(1, 1) to (4, 4). In this case, the pixel locations (1, 1), (4, 1), (1,4), and (4, 4) may be defined as vertices of the corresponding blockarea. In the block area, gain values respectively corresponding to thepixel locations (1, 1), (4, 1), (1, 4), and (4, 4) defined as verticesmay be the reference gain values.

The plurality of block areas according to an embodiment of the presentdisclosure may include a first block area and a second block area. Here,the second block area may have a shorter distance from the center of theimage thereto than the distance from the center of the image to thefirst block area (i.e., closer to the center of the image than the firstblock area), and may have a size greater than that of the first blockarea. In an embodiment, within an image, a size of a block area maybecome smaller as the block area is located farther from the center ofthe image. In an embodiment, within an image, a block area locatedclosest to the center of the image may have the greatest size and ablock area located closest to the edge of the image may have thesmallest size.

Also, the image processor 200 may calculate gain values included in eachof the plurality of block areas using the reference gain values. Here,the calculated gain values denote the remaining gain values other thanthe reference gain values included in each of the plurality of blockareas, and hereinafter each of the remaining gain values will bereferred to as a “gain value” for convenience of description.Accordingly, all gain values included in the gain map may be calculated.The image processor 200 may store the gain values included in the gainmap in the memory 300.

The memory 300 may store the reference gain values included in the gainmap. In an embodiment of the present disclosure, the memory 300 maystore the gain values included in the gain map.

The memory 300 may be implemented as a nonvolatile memory device. Forexample, the memory 300 may be implemented as any of various nonvolatilememory devices such as a read-only memory (ROM) that enables onlyreading of data, a one-time programmable (OTP) memory that enableswriting only once, an Erasable and Programmable ROM (EPROM) that enableserasure of stored data and writing of data, a NAND flash memory, and aNOR flash memory.

In accordance with an embodiment of the present disclosure, there can beprovided an image sensing device and a method of operating the imagesensing device, which may reduce a computational load and the amount ofdata to be used for image correction. Hereinafter, the presentdisclosure will be described in detail with reference to the attacheddrawings.

FIG. 2 is a diagram illustrating an image sensor according to anembodiment of the present disclosure.

Referring to FIG. 2 , an image sensor 100 may include an optical lensLS, a pixel array 110, a row decoder 120, a timing generator 130, asignal transducer 140, and an output buffer 150.

The optical lens LS may refract light that has been reflected from anobject and has reached the optical lens LS. The light refracted throughthe optical lens LS may travel toward the pixel array 110. The opticallens LS may be either one lens or a set of a plurality of lensesarranged in a light-travel path. Further, the optical lens LS mayinclude a set of micro-lenses arranged on the tops of respective pixelsof the pixel array 110. The object may include at least one of variouselements, such as an arbitrary item, an animal, a person, and abackground present outside of the image sensor 100.

The pixel array 110 may include a color filter array and a photoelectricconversion layer. The color filter array may be disposed on the top ofthe photoelectric conversion layer. The photoelectric conversion layermay be disposed on the bottom of the color filter array. Here, the topand the bottom may be defined based on the direction in which lighttravels, and light may travel in the direction from the color filterarray to the photoelectric conversion layer.

The color filter array may include a plurality of color filters. Forexample, each of the plurality of color filters may be one of a redcolor filter, a green color filter, and a blue color filter. The redcolor filter may transmit light having a wavelength corresponding to ared color by filtering incident light. The green color filter maytransmit light having a wavelength corresponding to a green color byfiltering the incident light. The blue color filter may transmit lighthaving a wavelength corresponding to a blue color by filtering theincident light. However, this is only an embodiment, and the type ofcolor filter, which transmits light of a specific color, may bevariously changed.

The photoelectric conversion layer may include a plurality of sensingcircuits. Each sensing circuit may include a photodiode and a capacitor.The photodiode may produce current corresponding to the light incidentthereon through a photoelectric effect. The capacitor may accumulatecharges depending on the current produced by the photodiode. Here, theamount of the accumulated charge may correspond to a pixel valueindicating brightness.

The row decoder 120 may select a pixel located in a row corresponding toan address in response to the address and control signals output fromthe timing generator 130. The pixel array 110 may output a signalcorresponding to the amount of accumulated charge from the selectedpixel and provide the signal to the signal transducer 140.

The signal transducer 140 may acquire pieces of pixel data about theplurality of pixels based on respective signals output from the pixelarray 110. Here, the pixel data may include a pixel value and a pixelcolor. When one pixel is described by way of example, the signaltransducer 140 may acquire a pixel value corresponding to the amount ofcharge, accumulated in the capacitor in the photoelectric conversionlayer, and a color corresponding to the color filter.

The output buffer 150 may output an image Img or a calibration imageCal_Img. In detail, the output buffer 150 may be implemented using aplurality of buffers which store digital signals output from the signaltransducer 140. The output buffer 150 may latch and output pieces ofpixel data in column units, which are provided from the signaltransducer 140. The output buffer 150 may temporarily store the piecesof pixel data output from the signal transducer 140, and maysequentially output the pieces of pixel data under the control of thetiming generator 130.

FIG. 3A is a diagram illustrating a pixel array according to anembodiment of the present disclosure.

Referring to FIG. 3A, the pixel array 110 according to an embodiment ofthe present disclosure may be implemented as a pixel array 310 having apixel structure such as that illustrated in FIG. 3A. The pixel array 310may include a plurality of pixels. The plurality of pixels may bearranged in a column direction and a row direction. Each of theplurality of pixels may include information about the location at whichthe corresponding pixel is arranged. For example, a pixel CE_xy mayindicate a pixel arranged at an x-th location in a column direction anda y-th location in a row direction. Here, each of x and y may be anatural number.

Each of the plurality of pixels may include a microlens ML whichrefracts light, a color filter CF which transmits light of a specificcolor, and a sensing circuit PD which detects the intensity of light.The plurality of pixels may be classified according to the type of colorfilter CF. For example, pixels including a red color filter may bereferred to as pixels with red channels R1 to R4, pixels including agreen color filter may be referred to as pixels with green channels Gr1to Gr4 and Gb1 to Gb4, and pixels including a blue color filter may bereferred to as pixels with blue channels B1 to B4.

The pixel array 310 may include a plurality of pixel groups. Theplurality of pixel groups may be arranged in a column direction and arow direction. Each of the plurality of pixel groups may includeinformation about the location at which the corresponding pixel group isarranged. For example, a pixel group CG_XY may indicate a pixel grouparranged at an X-th location in a column direction and a Y-th locationin a row direction. Here, each of X and Y may be a natural number.

Each pixel group may include a plurality of pixels arranged in a presetarray pattern. That is, the pixel group may indicate a unit area inwhich an array pattern of a plurality of pixels is repeated. Forexample, the preset array pattern may be a quad Bayer pattern in whichpixels with 2×2 first green channels Gb1 to Gb4, pixels with 2×2 bluechannels B1 to B4, pixels with 2×2 red channels R1 to R4, and pixelswith 2×2 second green channels Gr1 to Gr4 are arranged in a 4×4 array.

The above-described embodiment is only an embodiment, and the presetarray pattern may be modified into various patterns, such as a Bayerpattern in which a pixel with a 1×1 first green channel, a pixel with a1×1 blue channel, a pixel with a 1×1 red channel, and a pixel with a 1×1second green channel are arranged in a 2×2 array.

FIG. 3B is a diagram illustrating an image acquired through an imagesensor according to an embodiment of the present disclosure.

Referring to FIG. 3B, an image Img or a calibration image Cal_Imgaccording to an embodiment of the present disclosure may be an image 320having a pixel structure such as that illustrated in FIG. 3B.

The image 320 may be acquired through the image sensor 100 including thepixel array 310.

The image 320 may include a plurality of pixel data pieces. Theplurality of pixel data pieces included in the image 320 may correspondto a plurality of pixels included in the pixel array 310 of the imagesensor.

Each of the plurality of pixel data pieces may include information aboutthe location at which the corresponding pixel is arranged. For example,a pixel data piece PX_xy may indicate a pixel arranged at an x-thlocation in a column direction and a y-th location in a row direction.Here, each of x and y may be a natural number. The pixel data piecePX_xy may correspond to the pixel CE_xy having the same location as thepixel data piece PX_xy. The pixel data piece PX_xy may includeinformation such as a pixel value acquired from the pixel CE_xy.

The image 320 may include a plurality of pixel data groups. Theplurality of pixel data groups may be arranged in a column direction anda row direction. Each of the plurality of pixel data groups may includeinformation about the location at which the corresponding pixel datagroup is arranged. For example, a pixel data group PG_XY may indicate apixel data group arranged at an X-th location in a column direction anda Y-th location in a row direction. Here, each of X and Y may be anatural number.

Each pixel data group may include a plurality of pixel data piecesarranged in a preset array pattern. That is, the pixel data group mayindicate a unit area in which an array pattern of a plurality of pixeldata pieces is repeated. For example, the preset array pattern may be aquad Bayer pattern in which pixel data pieces with 2×2 first greenchannels Gb1 to Gb4, pixel data pieces with 2×2 blue channels B1 to B4,pixel data pieces with 2×2 red channels R1 to R4, and pixel data pieceswith 2×2 second green channels Gr1 to Gr4 are arranged in a 4×4 array.

The pixel data pieces included in the image 320 may be classifiedaccording to the color channel or the channel. For example, the pixeldata pieces included in the image 320 may be classified into pixel datapieces with first green channels Gb1 to Gb4, pixel data pieces with bluechannels B1 to B4, pixel data pieces with red channels R1 to R4, andpixel data pieces with second green channels Gr1 to Gr4 according to thecolor channel. Further, the pixel data pieces included in the image 320may be classified into pixel data pieces with first to fourth channelsGb1 to Gb4, pixel data pieces with fifth to eighth channels Gr1 to Gr4,pixel data pieces with ninth to twelfth channels R1 to R4, and pixeldata pieces with thirteenth to sixteenth channels B1 to B4 according tothe channel.

Meanwhile, the above-described embodiment is only an embodiment, and thepreset array pattern may be modified into various patterns, such as aBayer pattern in which a pixel data piece with a 1×1 first greenchannel, a pixel data piece with a 1×1 blue channel, a pixel data piecewith a 1×1 red channel, and a pixel data piece with a 1×1 second greenchannel are arranged in a 2×2 array.

FIG. 4 is a diagram illustrating an image processor according to anembodiment of the present disclosure.

Referring to FIG. 4 , the image processor 200 may include an imagecalibrator 210 and an image corrector 220.

The image calibrator 210 may calculate reference gain values ref_Gainusing a calibration image Cal_Img acquired through the image sensor 100.

In detail, the image calibrator 210 may select pixel data piecescorresponding to vertices of each block area from among a plurality ofpixel data pieces included in the calibration image Cal_Img acquiredthrough the image sensor 100. Hereinafter, an example will be describedin which a reference gain value ref_Gain which corresponds to one of thelocations of the selected pixel data pieces being calculated.

The image calibrator 210 may calculate the average pixel value of pixeldata pieces having the same color channel as the pixel data pieceselected from among the pixel data pieces included in a region ofinterest including the selected pixel data piece. Here, the same colorchannel may be one of a first green channel, a second green channel, ared channel, and a blue channel. In an embodiment, the median value ofpixel values of the pixel data pieces, instead of the average pixelvalue, may also be used.

The region of interest may be a region extending to a preset size basedon the locations of pixel data pieces corresponding to vertices of theblock area. In an embodiment, regions of interest may have the samesize. In other embodiments, the regions of interest may have differentsizes depending on the location of a pixel data piece that is areference.

As an example, the same color channel as the selected pixel data pieceis a first green channel.

In an embodiment, an average pixel value may be the average of the pixelvalues of pixel data pieces with the first green channel, among thepixel data pieces included in the region of interest. In otherembodiments, an average pixel value may be the average of the pixelvalues of pixel data pieces with the first green channel and the pixelvalues of pixel data pieces with the second green channel, among thepixel data pieces included in the region of interest. Even in the casewhere the color channel that is the same as that of the selected pixeldata piece is a second green channel, the average may be calculated inthe same manner as above.

The image calibrator 210 may calculate a reference gain value ref_Gaincorresponding to the selected pixel data piece using the pixel value ofthe selected pixel data piece and the calculated average pixel value.For example, the reference gain value ref_Gain may be the ratio of thepixel value of the selected pixel data piece to the calculated averagepixel value.

For the remaining pixel data pieces, among the selected pixel datapieces, the image calibrator 210 may calculate reference gain valuesref_Gain in such a manner. Further, the image calibrator 210 may storethe calculated reference gain values ref_Gain in the memory 300.

The image calibrator 210 may calculate gain values Gain using thereference gain values ref_Gain.

In detail, based on the distances from a point selected in one of aplurality of block areas to vertices of the one block area and referencegain values ref_Gain corresponding to the vertices of the one blockarea, the image calibrator 210 may calculate a gain value Gaincorresponding to the selected point. Here, the distances from thevertices to the selected point may be the distances from the pixel datapieces corresponding to the vertices to the pixel data piececorresponding to the selected point. The distances between pixel datapieces may correspond to the number of pixel data pieces present betweenthe corresponding pixel data pieces. The distance may be a conceptincluding at least one of a length in a column direction and a length ina row direction.

In an embodiment, the vertices of one block area may include a firstvertex, a second vertex, a third vertex, and a fourth vertex of the oneblock area.

In this case, the image calibrator 210 may calculate a gain value Gaincorresponding to the selected point based on the distance from theselected point to the first vertex, the distance from the selected pointto the second vertex, the distance from the selected point to the thirdvertex, the distance from the selected point to the fourth vertex, thereference gain value corresponding to the first vertex, the referencegain value corresponding to the second vertex, the reference gain valuecorresponding to the third vertex, and the reference gain valuecorresponding to the fourth vertex.

In an embodiment, the plurality of block areas may include a first blockarea and a second block area. Here, the second block area may have ashorter distance from the center of the image thereto than the distancefrom the center of the image to the first block area, and may have asize greater than that of the first block area. This uses a phenomenonin which the degree of the overall distortion is lower and the degreesof distortion for respective areas are more uniform in the direction tothe center of the image Img or the calibration image Cal_Img, whereasthe degree of the entire distortion is higher and the degrees ofdistortion for respective areas are not uniform in the direction to theedge of the image or the calibration image. When a block area having arelatively large size is set in the direction to the center of the imageImg or the calibration image Cal_Img and a block area having arelatively small size is set in the direction to the edge of the imageImg or the calibration image Cal_Img, the reference gain value ref_Gainincluded in the block area closer to the center may be calculated lessoften. The gain value Gain may be calculated more often, but the entirecomputational load may be reduced when the gain value Gain is calculatedusing an algorithm which imposes a lower computational load than thereference gain value ref_Gain.

In an embodiment, a larger number of bits than that of gain valuesincluded in the second block area may be assigned to gain valuesincluded in the first block area. Here, the distance from the center ofthe image to the second block area may be shorter than the distance fromthe center of the image to the first block area, and the second blockarea may have a size greater than that of the first block area. Thenumber of data bits to be assigned to each gain value may change withthe location or area. The larger a number of bits assigned to data, thewider a value range of the data is. For example, when calculating a gainvalue to which a small number of bits are assigned, the calculated gainvalue may be expressed within a small value range allowed by the smallnumber of bits.

When the image Img is acquired through the image sensor 100, the imagecorrector 220 may generate a correction image Cor_Img in which thereference gain values ref_Gain and the gain values Gain that are storedin the memory 300 are applied to the plurality of pixel data piecesincluded in the image. Here, the reference gain values ref_Gain and thegain values Gain may be included in the gain map. The reference gainvalues ref_Gain and the gain values Gain may correspond to therespective pixel data pieces included in the image.

The image corrector 220 may output the correction image Cor_Img. Theimage corrector 220 may output the correction image Cor_Img to the host3000 or the device indicated by the host 3000. For example, the imagecorrector 220 may output the correction image Cor_Img to a processor, adisplay, or a storage in response to a control command received from thehost 3000.

In an embodiment, the image corrector 220 may generate the correctionimage Cor_Img using the reference gain values ref_Gain output from thememory 300 and the gain values Gain output from the image calibrator 210or a volatile memory. That is, the gain values Gain may not be stored inthe memory 300.

For this, whenever a preset event occurs, the image calibrator 210 maycalculate gain values Gain using the reference gain values ref_Gainstored in the memory 300. Here, the preset event may include an eventthat allows the image sensor 100 to be turned on. The image calibrator210 may transfer the calculated gain values Gain to the image corrector220. Alternatively, the image calibrator 210 may store the calculatedgain values Gain in the volatile memory. The volatile memory may beimplemented as a Static Random Access Memory (SRAM), a Dynamic RAM(DRAM) or the like.

In an embodiment, the image corrector 220 may generate the correctionimage Cor_Img using the reference gain values ref_Gain and the gainvalues Gain, which are output from the memory 300. That is, the gainvalues Gain may be stored, together with the reference gain valuesref_Gain, in the memory 300.

FIG. 5 is a diagram illustrating a calibration image according to anembodiment of the present disclosure.

Referring to FIG. 5 , an image sensor 100 may acquire an image 500. Forexample, the image 500 may be a calibration image Cal_Img acquired bythe image sensor 100 to capture a white background in an environmenthaving uniform illuminance. The image 500 may indicate the state beforecorrection is performed. The shading of each portion of the image 500may indicate the brightness of each pixel data piece or the pixel valueof each pixel data piece. It can be seen that the difference in shadingis increased in the direction from the center C of the image 500 to theedge of the image 500 farther away from the center C. It can be seenthat the degree of distortion is further increased and is morenon-uniform in the direction from the center C of the image 500 to theedge of the image 500.

In accordance with an embodiment of the present disclosure, a pluralityof block areas Blk1 and Blk2 may be set to correct such distortion. Thesize of each of the plurality of block areas Blk1 and Blk2 may vary withthe distance from the center C of the image 500 to the correspondingblock area.

In an embodiment, the size of each of the plurality of block areas Blk1and Blk2 may decrease as the distance from the center C of the image 500to the corresponding block area is larger. Here, the center C of theimage 500 may be the point at which the length of the image 500 in acolumn direction and the length of the image 500 in a row direction arehalved. For example, the distance between the first block area Blk1 andthe center C of the image 500 may be defined as a distance d1 betweenthe center c1 of the first block area Blk1 and the center C of the image500. In the same manner, the distance between the second block area Blk2and the center C of the image 500 may be defined as a distance d2between the center c2 of the second block area Blk2 and the center C ofthe image 500.

The above-described embodiment is only an embodiment, and a referencelocation that is the criterion for determination of the size of eachblock area may be changed to various locations such as the vertices ofthe image 500, in addition to the center C of the image 500. Also, thecriterion for determining the size of each block area may be the ratioof the distance in a column direction to the distance in a rowdirection. For example, when a description is made based on the firstblock area Blk1, the ratio of the distances in the column direction maybe a first ratio, obtained by dividing the distance b1 in the columndirection between the center c1 of the first block area Blk1 and thecenter C of the image 500 by the length of the image 500 in the columndirection. The ratio of the distances in the row direction may be asecond ratio, obtained by dividing the distance a1 in the row directionbetween the center c1 of the first block area Blk1 and the center C ofthe image 500 by the length of the image 500 in the row direction.Further, when the first ratio and the second ratio are greater thanpreset values, the size of the corresponding block area may be set to afirst size, whereas when the first ratio and the second ratio are lessthan the preset values, the size of the corresponding block area may beset to a second size greater than the first size.

FIG. 6 is a diagram illustrating a gain map according to an embodimentof the present disclosure.

Referring to FIG. 6 , a gain map 600 may correspond to the size of animage Img acquired from the image sensor 100 or a calibration imageCal_Img.

The gain map 600 may include a plurality of block areas Blk1, Blk2, andBlk3. In an embodiment, each of the plurality of block areas Blk1, Blk2,and Blk3 may be a square or rectangular area. Here, respective blockareas Blk1, Blk2, and Blk3 may be logically defined areas, and each ofthe block areas Blk1, Blk2, and Blk3 may correspond to one area of theimage Img or the calibration image Cal_Img.

The plurality of block areas Blk1, Blk2, and Blk3 may include the firstblock area Blk1, the second block area Blk2, and the third block areaBlk3. The first block area Blk1, the second block area Blk2, and thethird block area Blk3 may be arranged to correspond to the image Img orthe calibration image Cal_Img. The first block area Blk1, the secondblock area Blk2, and the third block area Blk3 may be block areas havingdifferent sizes. The locations at which the first block area Blk1, thesecond block area Blk2, and the third block area Blk3 are arranged mayvary with the sizes thereof. Alternatively, the sizes of the first blockarea Blk1, the second block area Blk2, and the third block area Blk3 mayvary with the locations at which the block areas Blk1, Blk2, and Blk3are arranged.

For example, the third block Blk3, having the largest size, may bearranged at the location at which the distance to the center of theimage is the shortest. The second block area Blk2, having anintermediate size, may be arranged at the location at which the distanceto the center of the image falls into an intermediate range. The firstblock area Blk1, having the smallest size, may be arranged at thelocation at which the distance to the edge of the image is the shortest.

The image processor 200 may calculate reference gain values ref_Gainusing the pixel values of the pixel data pieces included in thecalibration image Cal_Img and the block areas Blk1, Blk2, and Blk3.

In detail, the image processor 200 may select pixel data pieces atpreset locations from among the plurality of pixel data pieces includedin the calibration image Cal_Img. Here, the pixel data pieces at thepreset locations may be pixel data pieces corresponding to the verticesof the block areas Blk1, Blk2, and Blk3. The pixel data pieces at thepreset locations may be coupled through lines in a column direction anda row direction, and thus the block areas Blk1, Blk2, and Blk3 may bedefined. One reference pixel data group ref_PG_XY may be one pixel datagroup PG_XY located at the location (X, Y) indicating one vertex of oneblock area from among the plurality of pixel data groups. One pixel datagroup PG_XY may include pixel data pieces with different color channels.The reference pixel data group ref_PG_XY may be located at the vertex ofat least one block area Blk2 or Blk3. A vertex of one block area may bea vertex of another block area. That is, the block areas adjacent toeach other may share at least one vertex.

The image processor 200 may select the regions of interest (ROI)corresponding to respective vertices of the block areas Blk1, Blk2, andBlk3. Each region of interest (ROI) may be an area having a preset sizeand having a reference pixel data group ref_PG_XY (i.e., a vertex) as areference point. Here, the reference point may be a central point of aregion of interest ROI. The preset size may be a fixed size. However,this is only an embodiment, and the preset size may be variableaccording to the location of the reference pixel data group ref_PG_XY(i.e., the vertex). The region of interest (RIO) may include pixel datapieces located in the corresponding area. A detailed embodiment thereofwill be described below with reference to FIG. 7 .

FIG. 7 is a diagram illustrating any one block area according to anembodiment of the present disclosure.

Referring to FIG. 7 , the image processor 200 may select regions ofinterest ROI1 to ROI4 corresponding to respective vertices Ref_PG1 toRef_PG4 of one block area BLKi.

For example, the block area BLKi may have a square or rectangular shape.The length of the block area BLKi in a column direction may be W, andthe length thereof in a row direction may be H. Here, W and H may benatural numbers that are equal to or different from each other. Thevertices Ref_PG1 to Ref_PG4 of the block area BLKi may include a firstvertex Ref_PG1, a second vertex Ref_PG2, a third vertex Ref_PG3, and afourth vertex Ref_PG4. The first vertex Ref_PG1, the second vertexRef_PG2, the third vertex Ref_PG3, and the fourth vertex Ref_PG4 may bedisposed at a distance of W from each other in a column direction and ata distance of H in a row direction.

In this case, the image processor 200 may select a first region ofinterest ROI1 having a preset size that uses the location of the firstvertex Ref_PG1 as a central point, may select a second region ofinterest ROI2 having a preset size that uses the location of the secondvertex Ref_PG2 as a central point, may select a third region of interestROI3 having a preset size that uses the location of the third vertexRef_PG3 as a central point, and may select a fourth region of interestROI4 having a preset size that uses the location of the fourth vertexRef_PG4 as a central point.

The image processor 200 may calculate reference gain values ref_Gaincorresponding to the plurality of vertices Ref_PG1 to Ref_PG4 using thepixel values of the pixel data pieces included in respective areascorresponding to the plurality of regions of interest ROI1 to ROI4 inthe calibration image Cal_Img. That is, the reference gain valuesref_Gain of each of the plurality of vertices Ref_PG1 to Ref_PG4 may becalculated using pixel values included in each of the plurality ofvertices and neighboring pixel values.

For example, the image processor 200 may calculate a reference gainvalue corresponding to the first vertex Ref_PG1 of the block area BLKiusing the pixel values of the pixel data pieces included in the areacorresponding to the first region of interest ROI1 in the calibrationimage Cal_Img. The image processor 200 may calculate a reference gainvalue corresponding to the second vertex Ref_PG2 of the block area BLKiusing the pixel values of the pixel data pieces included in the areacorresponding to the second region of interest ROI2 in the calibrationimage Cal_Img. The image processor 200 may calculate a reference gainvalue corresponding to the third vertex Ref_PG3 of the block area BLKiusing the pixel values of the pixel data pieces included in the areacorresponding to the third region of interest ROI3 in the calibrationimage Cal_Img. The image processor 200 may calculate a reference gainvalue corresponding to the fourth vertex Ref_PG4 of the block area BLKiusing the pixel values of the pixel data pieces included in the areacorresponding to the fourth region of interest ROI4 in the calibrationimage Cal_Img.

The image processor 200 may calculate the remaining gain values Gainincluded in the block area BLKi using the reference gain values ref_Gaincorresponding to the plurality of vertices Ref_PG1 to Ref_PG4.Hereinafter, a detailed embodiment in which reference gain values arecalculated will be described with reference to FIGS. 8A to 8D.

FIGS. 8A to 8D are diagrams illustrating a method of calculatingreference gain values according to an embodiment of the presentdisclosure.

Referring to FIG. 8A, the image processor 200 may calculate referencegain values ref_Gain_xy using Equations (1-1) to (3).

In Equations (1-1) to (3) of FIG. 8A, a pixel value ref_PV_xy indicatesthe pixel value of a selected pixel data piece. The selected pixel datapiece represents one pixel data piece selected from among the pixel datapieces included in a pixel data group PG_XY having the same arrangementlocations as those of the vertices.

The case where the selected pixel data piece is one of pixel data pieceswith first green channels Gb1 to Gb4 is described.

In an embodiment, as shown in Equation (1-1) of FIG. 8A, the imageprocessor 200 may calculate the reference gain value ref_Gain_xy usingthe pixel value ref_PV_xy of the selected pixel data piece and theaverage pixel value ROI_AvgPV_G of the green channels Gb1 to Gb4 and Gr1to Gr4. Here, the average pixel value ROI_AvgPV_G of the green channelsGb1 to Gb4 and Gr1 to Gr4 may be the average of the pixel values of thepixel data pieces with the first green channels Gb1 to Gb4 and the pixelvalues of the pixel data pieces with the second green channels Gr1 toGr4, among the plurality of pixel data pieces included in thecorresponding region of interest (ROI).

In an embodiment, as shown in Equation (1-2) of FIG. 8A, the imageprocessor 200 may calculate the reference gain value ref_Gain_xy usingthe pixel value ref_PV_xy of the selected pixel data piece and theaverage pixel value ROI_AvgPV_Gb of the first green channels Gb1 to Gb4.Here, the average pixel value ROI_AvgPV_Gb of the first green channelsGb1 to Gb4 may be the average of the pixel values of the pixel datapieces with the first green channels Gb1 to Gb4, among the plurality ofpixel data pieces included in the region of interest (ROI).

The case where the selected pixel data piece is one of pixel data pieceswith second green channels Gr1 to Gr4 is described.

In an embodiment, as shown in Equation (1-1) of FIG. 8A, the imageprocessor 200 may calculate the reference gain value ref_Gain_xy usingthe pixel value ref_PV_xy of the selected pixel data piece and theaverage pixel value ROI_AvgPV_G of the green channels Gb1 to Gb4 and Gr1to Gr4.

In an embodiment, as shown in Equation (1-3) of FIG. 8A, the imageprocessor 200 may calculate the reference gain value ref_Gain_xy usingthe pixel value ref_PV_xy of the selected pixel data piece and theaverage pixel value ROI_AvgPV_Gr of the second green channels Gr1 toGr4. Here, the average pixel value ROI_AvgPV_Gr of the second greenchannels Gr1 to Gr4 may be the average of the pixel values of the pixeldata pieces with the second green channels Gr1 to Gr4, among theplurality of pixel data pieces included in the region of interest (ROI).

The case where the selected pixel data piece is one of pixel data pieceswith red channels R1 to R4 is described. In this case, as shown inEquation (2) of FIG. 8A, the image processor 200 may calculate thereference gain value ref_Gain_xy using the pixel value ref_PV_xy of theselected pixel data piece and the average pixel value ROI_AvgPV_R of thered channels R1 to R4. The average pixel value ROI_AvgPV_R of the redchannels R1 to R4 may be the average of the pixel values of the pixeldata pieces with the red channels R1 to R4, among the plurality of pixeldata pieces included in the region of interest (ROI).

The case where the selected pixel data piece is one of pixel data pieceswith blue channels B1 to B4 is described. In this case, as shown inEquation (3) of FIG. 8A, the image processor 200 may calculate thereference gain value ref_Gain_xy using the pixel value ref_PV_xy of theselected pixel data piece and the average pixel value ROI_AvgPV_B of theblue channels B1 to B4. The average pixel value ROI_AvgPV_B of the bluechannels B1 to B4 may be the average of the pixel values of the pixeldata pieces with the blue channels B1 to B4, among the plurality ofpixel data pieces included in the region of interest (ROI).

Hereinafter, a method of calculating one reference gain value will bedescribed with reference to FIGS. 8B and 8C.

Referring to FIGS. 8B and 8C, the image processor 200 may select onepixel data piece from among pixel data pieces in a pixel data groupPG_XY having the same arrangement location as that of a correspondingvertex of a block area Blki in a calibration image 810 or 820. Here, apixel data piece having a pixel value ref_PX_xy with a first channel Gb1disposed at (x, y) is selected.

The image processor 200 may select a region of interest (ROI), obtainedby extending the pixel data group PG_XY as a central area to a presetsize, in the calibration image 810 or 820.

In an embodiment, as illustrated in FIG. 8B, the image processor 200 mayselect pixel data pieces ROI_Avg_Gb with first green channels Gb1 to Gb4which are the color channels identical to that of the selected pixeldata piece having the pixel value ref_PX_xy, from among the plurality ofpixel data pieces included in the region of interest (ROI) in the image810. The image processor 200 may calculate an average pixel valueROI_AvgPV_Gb which is the average of the pixel values of the selectedpixel data pieces ROI_Avg_Gb. As shown in Equation (1-2) of FIG. 8A, theimage processor 200 may calculate a reference gain value ref_Gain_xy forthe location (x, y) using the pixel value ref_PV_xy of the pixel datapiece selected at the location (x, y) and the average pixel valueROI_AvgPV_Gb. In this manner, reference gain values for other locationsmay be calculated. In an embodiment, the pixel value ref_PV_xy in theEquation (1-2) of FIG. 8A may be replaced with an average valueTOT_Avg_Gb1 of Gb1 pixel values included in the ROI.

In an embodiment, as illustrated in FIG. 8C, the image processor 200 mayselect pixel data pieces ROI_Avg_G with first green channels Gb1 to Gb4and second green channels Gr1 to Gr4 which are the type of colorchannels identical to that of the selected pixel data piece having thepixel value ref_PX_xy, from among the plurality of pixel data piecesincluded in the region of interest (ROI) in the image 820. The imageprocessor 200 may calculate an average pixel value ROI_AvgPV_G which isthe average of the pixel values of the selected pixel data piecesROI_Avg_G. As shown in Equation (1-1) of FIG. 8A, the image processor200 may calculate a reference gain value ref_Gain_xy for the location(x, y) using the pixel value ref_PV_xy of the pixel data piece selectedat the location (x, y) and the average pixel value ROI_AvgPV_G. In thismanner, reference gain values for other locations may be calculated. Inan embodiment, the pixel value ref_PV_xy in the Equation (1-1) of FIG.8A may be replaced with the average value TOT_Avg_Gb1 of Gb1 pixelvalues included in the ROI.

Referring back to FIG. 6 , the first blocks Blk1 having the smallestsize, among the plurality of blocks Blk1, Blk2, and Blk3, may bearranged in an edge area 630. In accordance with an embodiment of thepresent disclosure, in the case of the first blocks Blk1 arranged in theedge area 630, the reference gain value of the first blocks Blk1 may becalculated through a mirroring method. This will be described below withreference to FIG. 8D.

Referring to FIG. 8D, in an edge area 830, first blocks Blk1 having thesmallest size may be arranged, and a plurality of vertices 850 and 860of the first blocks Blk1 may include a first vertex 850 having a shorterdistance from the center of an image (i.e., closer to the center) than asecond vertex 860. The second vertex 860 may be closest to the firstvertex 850 among all vertices of all block areas. That is, the secondvertex 860 may be a vertex disposed in the outermost location at theedge of the image, among all of the vertices.

The image processor 200 may calculate first reference gain valuescorresponding to the first vertex 850 closer to the center. In detail,the image processor 200 may select the reference pixel data groupref_PG_XY corresponding to the location (X, Y) of the first vertex 850from the calibration image Cal_Img and may select a region of interest840 based on the reference pixel data group ref_PG_XY. The imageprocessor 200 may calculate first reference gain values corresponding tolocation (X, Y) using the pixel values ref_PV_xy of the pixel datapieces included in the reference pixel data group ref_PG_XY and thepixel data pieces included in the region of interest 840.

Further, the image processor 200 may copy the first reference gain valuecorresponding to the first vertex 850 to a second reference gain valuecorresponding to the second vertex 860. That is, the second referencegain value of the second vertex 860 may be the same values as the firstreference gain value of the first vertex 850.

FIGS. 9A and 9B are diagrams illustrating a method of calculating gainvalues according to an embodiment of the present disclosure.

Referring to FIGS. 9A and 9B, one block area Blki may be defined by aplurality of vertices. The one block area Blki may include thereinpoints having a specific location. One point may correspond to one pixelhaving the same location.

The image processor 200 may calculate gain values respectivelycorresponding to a plurality of points included in one block area Blkiusing reference gain values ref_PX1 to ref_PX4 corresponding to aplurality of vertices that define the one block area Blki. In this case,the image processor 200 may calculate the gain values of the samechannel using the reference gain values ref_PX1 to ref_PX4 correspondingto the plurality of vertices of the same channel. Here, a method ofcalculating a gain value Gain_P_xy corresponding to one point P_xy,among the plurality of points included in one block area Blki, isdescribed. Here, the channel is a first channel Gb1.

The image processor 200 may acquire a distance W in a column directionand a distance H in a row direction between the plurality of verticesdefining one block area Blki. For example, the image processor 200 mayacquire the distance W in the column direction and the distance H in therow direction based on the differences between the locations of theplurality of vertices.

The image processor 200 may acquire distances x1 and x2 in the columndirection and distances y1 and y2 in the row direction betweenrespective vertices defining one block area Blki and one selected pointP_xy. For example, the image processor 200 may acquire the distances x1and x2 in the column direction and the distances y1 and y2 in the rowdirection based on the differences between the locations of therespective vertices and one selected point P_xy.

The image processor 200 may calculate the gain value Gain_P_xycorresponding to one point P_xy using the reference gain values ref_PX1to ref_PX4 respectively corresponding to the plurality of vertices, thedistances x1, x2, and W in the column direction, and the distances y1,y2, and H in the row direction based on the equation shown in FIG. 9A.

FIG. 10 is a diagram illustrating a correction image according to anembodiment of the present disclosure.

Referring to FIG. 10 , the image processor 200 may generate a correctionimage Cor_Img by applying a gain map (Gain map) to an image Img acquiredthrough the image sensor 100.

In detail, the image processor 200 may multiply the pixel value I_xy ofa pixel data piece at a location (x, y), selected from among a pluralityof pixel data pieces included in the image Img, by a gain value Gain_xycorresponding to the selected location (x, y), among a plurality ofreference gain values and a plurality of gain values which are includedin the gain map. In this case, the image processor 200 may generate acorrection image Cor_Img having result values O_xy, obtained bymultiplying pixel values I_xy by respective gain values Gain_xy forrespective locations, as pixel values.

That is, the pixel values of the plurality of pixel data pieces includedin the correction image Cor_Img may be the result values obtained bymultiplying the pixel values of the plurality of pixel data pieces,which are included in the image Img, by the plurality of reference gainvalues and the plurality of gain values, which are included in the gainmap, for respective corresponding locations.

FIG. 11 is a flowchart illustrating a method of operating an imagesensing device according to an embodiment of the present disclosure.

Referring to FIG. 11 , the method of operating the image sensing device1000 according to an embodiment of the present disclosure may acquire acalibration image Cal_Img through the image sensor 100 at operationS1110.

Reference gain values corresponding to vertices of each of the pluralityof block areas Blk1, Blk2, and Blk3 in the calibration image Cal_Img maybe calculated at step S1120. That is, the reference gain valuescorresponding to the vertices defined by the plurality of block areasBlk1, Blk2, and Blk3 included in the gain map corresponding to the sizeof the calibration image Cal_Img may be calculated. In an embodiment,each of the plurality of block areas Blk1, Blk2, and Blk3 may be arectangular or square area.

Here, the plurality of block areas Blk1, Blk2, and Blk3 may include afirst block area Blk1 and a second block area Blk2, which is closer tothe center of the calibration image Cal_Img than the first block areaBlk1 and has a size greater than that of the first block area Blk1.

In an embodiment, the operation of calculating the reference gain valuesmay include the operation of selecting pixel data pieces, correspondingto the vertices of the plurality of block areas Blk1, Blk2, and Blk3,from among a plurality of pixel data pieces included in the calibrationimage Cal_Img acquired through the image sensor 100, the operation ofcalculating the average pixel value of pixel data pieces with a selectedcolor channel from among pixel data pieces included in a region ofinterest (ROI) including one of the selected pixel data pieces, and theoperation of calculating one reference gain value, among reference gainvalues, using the pixel value of the selected pixel data piece and theaverage pixel value.

In an embodiment, the same color channel may be a color channelcorresponding to one of a first green channel, a second green channel, ared channel, and a blue channel.

In an embodiment, the same color channel may be a color channelcorresponding to one of the first green channel and the second greenchannel. In this case, the average pixel value may include the averageof the pixel values of pixel data pieces with the first green channeland the pixel values of pixel data pieces with the second green channel,among the pixel data pieces included in the region of interest.

Further, gain values corresponding to respective points included in theplurality of block areas Blk1, Blk2, and Blk3 may be calculated usingthe reference gain values at operation S1130.

In an embodiment, the operation of calculating the gain values mayinclude the operation of calculating a gain value corresponding to apoint selected in one block area Blki of the plurality of block areasBlk1, Blk2, and Blk3, based on distances from the selected point tovertices of the one block area Blki and the reference gain valuescorresponding to the vertices of the one block area Blki.

In an embodiment, the vertices of one block area Blki may include afirst vertex, a second vertex, a third vertex, and a fourth vertex ofthe one block area Blki. In this case, the operation of calculating thegain value corresponding to the selected point may be configured tocalculate the gain value corresponding to the selected point based onthe distance from the selected point to the first vertex, the distancefrom the selected point to the second vertex, the distance from theselected point to the third vertex, the distance from the selected pointto the fourth vertex, the reference gain value corresponding to thefirst vertex, the reference gain value corresponding to the secondvertex, the reference gain value corresponding to the third vertex, andthe reference gain value corresponding to the fourth vertex.

Further, the reference gain values and the gain values may be stored inthe memory 300 at operation S1140.

In an embodiment, a larger number of bits than that of gain valuesincluded in the second block area Blk2 may be assigned to gain valuesincluded in the first block area Blk1.

In an embodiment, the operating method of the image sensing device 1000may calculate, from a calibration image, reference gain values ofreference locations within each of block areas. The operating method maygenerate a gain map by calculating remaining gain values of remaininglocations within the block area based on the reference gain values ofthe block area, the gain map including the reference gain values and theremaining gain values corresponding to respective pixel data pieceswithin the block area. The operating method may generate a correctedimage by applying the gain map to an input image. Here, the calibrationimage and the input image may have a geometrically identical array ofpixel data. The block areas may be divided into two or more groupslocated in respective layer regions, which are radially layered withreference to a center of the calibration image. The block areas locatedin one of the layer regions may have an identical size. The block arealocated in the layer region relatively closer to the center may have agreater size than a block area located in the layer region relativelyfarther from the center.

In an embodiment, the reference locations may be vertices of the blockarea.

FIG. 12 is a flowchart illustrating a method of operating an imagesensing device according to an embodiment of the present disclosure.

Referring to FIG. 12 , the method of operating the image sensing device1000 according to an embodiment of the present disclosure may acquire animage Img through the image sensor 100 at operation S1210. When acapturing command is received from the host 3000, the image sensingdevice 1000 may acquire the image Img through the image sensor 100.

Further, a correction image Cor_Img in which reference gain values andgain values stored in the memory 300 are applied to a plurality of pixeldata pieces may be generated at operation S1220. The reference gainvalues and the gain values stored in the memory 300 may be included in again map. The gain map may correspond to the size of a calibration imageCal_Img or the image Img. The reference gain values and the gain valuesincluded in the gain map may correspond to the plurality of pixels.

Further, the correction image Cor_Img may be output at operation S1230.The image sensing device 1000 may output the correction image Cor_Img tothe host 3000. Alternatively, the image sensing device 1000 may outputthe correction image Cor_Img to a device indicated by the host 3000. Forexample, the correction image Cor_Img may be output to an externaldevice of the image sensing device 1000, such as a storage device, aprocessor, or a display.

FIG. 13 is a diagram illustrating a computing system including an imagesensing device according to the embodiment of the present disclosure.

Referring to FIG. 13 , a computing system 2000 may include an imagesensor 2010, a processor 2020, a storage device 2030, a memory device2040, an input/output device 2050, and a display device 2060. Althoughnot illustrated in FIG. 13 , the computing system 2000 may furtherinclude a port through which the computing system 2000 is capable ofcommunicating with the storage device 2030, the memory device 2040, theinput/output (I/O) device 2050, the display device 2060 or the like, orwhich is capable of communicating with an external device.

The image sensor 2010 may acquire an image or a calibration image. Theimage sensor 2010 may store a gain map. The image sensor 2010 maygenerate a correction image Cor_Img in which the gain map is applied tothe acquired image. The image sensor 2010 may be coupled to theprocessor 2020 through an address bus, a control bus, and a data bus oran additional communication link different from the buses, and may thencommunicate with the processor 2020. Here, the description of theabove-described image sensing device 1000 may be applied to the imagesensor 2010.

The image sensor 2010 may be implemented in various types of packages.For example, at least some components of the image sensor 2010 may beimplemented using any of packages such as Package on Package (PoP), Ballgrid arrays (BGAs), Chip scale packages (CSPs), Plastic Leaded ChipCarrier (PLCC), Plastic Dual In Line Package (PDIP), Die in Waffle Pack,Die in Wafer Form, Chip On Board (COB), Ceramic Dual In Line Package(CERDIP), Plastic Metric Quad Flat Pack (MQFP), Thin Quad Flatpack(TQFP), Small Outline Integrated Circuit (SOIC), Shrink Small OutlinePackage (SSOP), Thin Small Outline Package (TSOP), System In Package(SIP), Multi Chip Package (MCP), Wafer-level Fabricated Package (WFP),and Wafer-Level Processed Stack Package (WSP). In accordance with anembodiment, the image sensor 2010 and the processor 2020 may beintegrated into a single chip, or may be integrated into differentchips, respectively.

The processor 2020 may control the overall operation of the computingsystem 2000. The processor 2020 may control the display device 2060 sothat the correction image Cor_Img is displayed. The processor 2020 maystore the correction image Cor_Img in the storage device 2030. Here, theabove description of the host 3000 may be applied to the processor 2020.

The processor 2020 may perform specific calculations or tasks. Inaccordance with an embodiment of the present disclosure, the processor2020 may include at least one of a central processing unit (CPU), anapplication processing unit (APU), and a graphics processing unit (GPU).

The processor 2020 may be coupled to the storage device 2030, the memorydevice 2040, and the input/output device 2050 through the address bus,the control bus, and the data bus, and may then communicate with thedevices. In accordance with an embodiment of the present disclosure, theprocessor 2020 may also be coupled to an expansion bus such as aPeripheral Component Interconnect (PCI) bus.

The storage device 2030 may store data such as the correction imageCor_Img. Here, the data stored in the storage device 2030 may beretained not only when the computing system 2000 is driven but also whenthe computing system 2000 is not driven. For example, the storage device2030 may be implemented as at least one of all types of nonvolatilememory devices such as a flash memory device, a solid state drive (SSD),a hard disk drive (HDD), and an optical disk.

The memory device 2040 may store data such as the correction imageCor_Img. Here, the data stored in the memory device 2040 may be retainedonly when the computing system 2000 is driven. Alternatively, the datastored in the memory device 2040 may be retained when the computingsystem 2000 is driven or not driven. For example, the memory device 2040may include a volatile memory device such as a Dynamic Random AccessMemory (DRAM) or a Static Random Access Memory (SRAM), and a nonvolatilememory device such as an Erasable Programmable Read-Only Memory (EPROM),an Electrically Erasable Programmable Read-Only Memory (EEPROM), or aflash memory device.

The input/output device 2050 may include an input device and an outputdevice. The input device may be a device that is capable of inputting auser's command through interaction, and may be implemented as, forexample, a keyboard, a keypad, a mouse, or a microphone. The outputdevice may be a device that is capable of outputting data, and may beimplemented as, for example, a printer or a speaker.

The display device 2060 may be a device that visually outputs data, suchas the correction image. For this operation, the display device 2060 maybe implemented as any of various types of display devices, such as aLiquid Crystal Display (LCD) which uses a separate backlight unit (e.g.,a light emitting diode (LED) or the like) as a light source and controlsthe degree of transmission of light (e.g., brightness or intensity oflight) emitted from a backlight unit through liquid crystal bycontrolling the molecular arrangement of the liquid crystal, and adisplay device which uses a self-emissive element (e.g., a mini LEDhaving a size of 100 to 200 um, a micro LED having a size of 100 um orless, an Organic LED (OLED), a Quantum dot LED (QLED), or the like) as alight source without requiring a separate backlight unit or liquidcrystal.

The display device 2060 may include a plurality of pixels. The pluralityof pixels of the display device 2060 may correspond in location to theplurality of pixel data pieces of the correction image Cor_Img. Theplurality of pixels of the display device 2060 may emit light withluminance values corresponding to respective pixel values of theplurality of pixel data pieces of the correction image Cor_Img, and thusan image may be displayed. The display device 2060 may include aplurality of driving circuits corresponding to a plurality of pixels.Here, each driving circuit may be implemented in a form such as anamorphous silicon (a-Si) thin film transistor (TFT), a low temperaturepoly silicon (LTPS) TFT, or an organic TFT (OTFT).

In an embodiment, the display device 2060 may be implemented as aflexible display having characteristics in which the shape of thedisplay device 2060 is bendable and restorable. In an embodiment, thedisplay device 2060 may be implemented as a transparent display havinglight-transmissive characteristics. In an embodiment, the display device2060 may be implemented as a touch display after being combined with atouch sensor for identifying a location touched by the user.

The present disclosure may provide an image sensing device and a methodof operating the image sensing device, which may reduce thecomputational load and the amount of data to be used for imagecorrection.

While the present disclosure has been illustrated and described withrespect to specific embodiments and drawings, the disclosed embodimentsare not intended to be restrictive. Further, it is noted that thepresent disclosure may be achieved in various ways through substitution,change, and modification, as those skilled in the art will recognize inlight of the present disclosure, without departing from the spiritand/or scope of the present disclosure and the following claims.Furthermore, the embodiments may be combined to form additionalembodiments

What is claimed is:
 1. An image sensing device, comprising: an imagesensor configured to acquire an image including a plurality of pixelvalues; a memory configured to store reference gain values correspondingto vertices of each of a plurality of block areas included in the image;and an image processor configured to: calculate at least one gain valueincluded in each of the plurality of block areas based on distancesbetween the at least one gain value and the reference gain values, andoutput a correction image comprising a correction pixel value obtainedby correcting a selected pixel value among the plurality of pixel valuesbased on a selected gain value corresponding to the selected pixel valueamong the reference gain values and the at least one gain value, whereinthe plurality of block areas include a first block area and a secondblock area having a shorter distance from a center of the image than thefirst block area and having a size greater than that of the first blockarea.
 2. The image sensing device according to claim 1, wherein a largernumber of bits are assigned to each gain value included in the firstblock area than a number of bits assigned to each gain value included inthe second block area.
 3. The image sensing device according to claim 1,wherein the image processor comprises an image calibrator configured to:select pixel values, corresponding to the vertices of each of theplurality of block areas, from among a plurality of pixel valuesincluded in a calibration image acquired through the image sensor,calculate an average pixel value of pixel values, having a color channelidentical to a color channel of one of the selected pixel values, amongpixel values included in a region of interest including each of theselected pixel values, and calculate one reference gain value among thereference gain values using a pixel value of each of the selected pixelvalues and the average pixel value.
 4. The image sensing deviceaccording to claim 3, wherein the identical color channel is one of afirst green channel, a second green channel, a red channel, and a bluechannel.
 5. The image sensing device according to claim 3, wherein: theidentical color channel is one of a first green channel and a secondgreen channel, and the average pixel value includes an average of pixelvalues with the first green channel and pixel values with the secondgreen channel, among the pixel values included in the region ofinterest.
 6. The image sensing device according to claim 1, wherein theimage processor comprises an image calibrator configured to calculatethe gain value corresponding to a point selected in one of the pluralityof block areas based on distances from the selected point to vertices ofthe one block area and the reference gain values corresponding to thevertices of the one block area.
 7. The image sensing device according toclaim 6, wherein: the vertices of the one block area include a firstvertex, a second vertex, a third vertex, and a fourth vertex of the oneblock area, and the image calibrator calculates the gain valuecorresponding to the selected point based on a distance from theselected point to the first vertex, a distance from the selected pointto the second vertex, a distance from the selected point to the thirdvertex, a distance from the selected point to the fourth vertex, thereference gain value corresponding to the first vertex, the referencegain value corresponding to the second vertex, the reference gain valuecorresponding to the third vertex, and the reference gain valuecorresponding to the fourth vertex.
 8. The image sensing deviceaccording to claim 1, wherein the image processor is further configuredto store the reference gain values and the at least one gain valueincluded in each of the plurality of block areas in the memory.
 9. Theimage sensing device according to claim 8, wherein the image processorcomprises an image corrector configured to generate, when the image isacquired through the image sensor, the correction image in which thereference gain values and the at least one gain value that are stored inthe memory are applied to the plurality of pixel values included in theimage.
 10. The image sensing device according to claim 1, wherein eachof the plurality of block areas is a rectangular or square area.
 11. Amethod of operating an image sensing device, comprising: calculatingreference gain values corresponding to vertices of each of a pluralityof block areas included in a calibration image acquired through theimage sensing device; calculating at least one gain value included ineach of the plurality of block areas based on distances between the atleast one gain value and the reference gain values; storing thereference gain values and the at least one gain value included in eachof the plurality of block areas in a memory; and when an image isacquired through the image sensing device, outputting a correction imagecomprising a correction pixel value obtained by correcting a selectedpixel value among the plurality of pixel values based on a selected gainvalue corresponding to the selected pixel value among the reference gainvalues and the at least one gain value, wherein the plurality of blockareas include a first block area and a second block area having ashorter distance from a center of the calibration image than the firstblock area and having a size greater than that of the first block area.12. The method according to claim 11, wherein a larger number of bitsare assigned to each gain value included in the first block area than anumber of bits assigned to each gain value included in the second blockarea.
 13. The method according to claim 11, wherein the calculating thereference gain values comprises: selecting pixel values, correspondingto the vertices of each of the plurality of block areas, from among aplurality of pixel values included in the calibration image acquiredthrough the image sensing device; calculating an average pixel value ofpixel values, having a color channel identical to a color channel of oneof the selected pixel values, among pixel values included in a region ofinterest including each of the selected pixel values; and calculatingone reference gain value among the reference gain values using a pixelvalue of each of the selected pixel values and the average pixel value.14. The method according to claim 13, wherein the identical colorchannel is one of a first green channel, a second green channel, a redchannel, and a blue channel.
 15. The method according to claim 13,wherein: the identical color channel is one of a first green channel anda second green channel, and the average pixel value includes an averageof pixel values of pixel values with the first green channel and pixelvalues of pixel values with the second green channel, among the pixelvalues included in the region of interest.
 16. The method according toclaim 11, wherein the calculating the at least one gain value gaincomprises calculating a gain value corresponding to a point selected inone of the plurality of block areas based on distances from the selectedpoint to vertices of the one block area and the reference gain valuescorresponding to the vertices of the one block area.
 17. The methodaccording to claim 16, wherein: the vertices of the one block areainclude a first vertex, a second vertex, a third vertex, and a fourthvertex of the one block area, and the calculating the gain valuecorresponding to the selected point comprises: calculating the gainvalue corresponding to the selected point based on a distance from theselected point to the first vertex, a distance from the selected pointto the second vertex, a distance from the selected point to the thirdvertex, a distance from the selected point to the fourth vertex, thereference gain value corresponding to the first vertex, the referencegain value corresponding to the second vertex, the reference gain valuecorresponding to the third vertex, and the reference gain valuecorresponding to the fourth vertex.
 18. The method according to claim11, wherein each of the plurality of block areas is a rectangular orsquare area.